Skip to content

[9.4] [CI] Speed up telemetry_check by pre-filtering to collector files (#265978)#267655

Merged
kibanamachine merged 1 commit into
elastic:9.4from
kibanamachine:backport/9.4/pr-265978
May 5, 2026
Merged

[9.4] [CI] Speed up telemetry_check by pre-filtering to collector files (#265978)#267655
kibanamachine merged 1 commit into
elastic:9.4from
kibanamachine:backport/9.4/pr-265978

Conversation

@kibanamachine
Copy link
Copy Markdown
Contributor

Backport

This will backport the following commits from main to 9.4:

Questions ?

Please refer to the Backport tool documentation

…astic#265978)

## Summary

The `telemetry.sh` quick check takes **~8m 45s** in CI — the single
slowest check in the `quick_checks` step. It also runs in the serial
`mayChangeFiles` queue, blocking all other serial checks behind it.

### The problem

`extractCollectors()` passes **all ~36K TS files** from each of the 10
telemetry roots into `ts.createProgram()`, even though only **~69
files** across the entire repo actually contain `makeUsageCollector` or
`makeStatsCollector` calls.

TypeScript parses and type-checks thousands of unrelated source files
that will never yield a collector.

### The fix

Pre-filter files with a simple regex scan on file contents before
feeding them to the TS compiler:

```ts
const collectorPaths = fullPaths.filter((p) => COLLECTOR_RE.test(readFileSync(p, 'utf-8')));
```

TypeScript still resolves all transitive imports needed for
type-checking the collector files, but avoids parsing thousands of
unrelated files.

### Benchmark results (local)

**`src/platform/` root** (largest OSS root):

| Approach | Root files | Source files TS loads | Time |
|---|---|---|---|
| Current (all files) | 8,852 | 17,279 | 48s |
| New (collector files only) | 30 | 4,814 | 15s |
| **Speedup** | | **3.6x fewer** | **3.1x faster** |

**`x-pack/platform/` root** (largest x-pack root):

| Approach | Root files | Source files TS loads | Time |
|---|---|---|---|
| Current (all files) | 15,344 | 25,002 | 71s |
| New (collector files only) | 27 | 13,835 | 49s |
| **Speedup** | | **1.8x fewer** | **1.5x faster** |

**CI estimate**: `telemetry.sh` should drop from **~8m 45s to ~3-4
min**.

### How it was tested

- Profiled `createKibanaProgram` locally with the actual Kibana
telemetry roots
- Verified the same collector files are found and processed
- The pre-filter is conservative — any file mentioning the collector
function names is included

Made with [Cursor](https://cursor.com)

Co-authored-by: Ahmad Bamieh <ahmad.bamyeh@elastic.co>
(cherry picked from commit b3cb3c8)
@kibanamachine
Copy link
Copy Markdown
Contributor Author

💚 Build Succeeded

Metrics [docs]

✅ unchanged

cc @shahzad31

@kibanamachine kibanamachine merged commit 26dc68c into elastic:9.4 May 5, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants